A light field (LF) camera, also known as a plenoptic camera, captures light travelling in every direction through a scene in four dimensions. That is, contrasted with a conventional camera, which records only light intensity, an LF camera captures both the intensities and the directions of the light rays. This enables sophisticated data processing even after the image data is captured. One can for example virtually change focus or perspective, or estimate depth maps from a single exposure.
By utilizing the rich information of LF images, and employing features of light fields (for example, regular sampling pattern, subpixel disparity, etc.), an LF camera can be used to generate a three-dimensional (3D) model of an object. For an accurate 3D model reconstruction, structure from motion (i.e., a method of solving LF poses) is an essential step. Traditional structure from motion methods often cannot achieve good results, since they often fail to use geometric constrains embedded in the light fields.
This disclosure presents a method of 3D model reconstruction which uses a new structure from motion method. The proposed structure from motion method exploits how ray geometry transforms under LF pose variations. This 3D model reconstruction method is more robust and more accurate, and can even work on topologically complex objects, such as flowers and so on. In addition, the 3D models generated by this method can be adopted in virtual reality (VR) devices and augmented reality (AR) devices, and can be viewed from any viewpoint.